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1.0 Introduction 


1.1 IRSS System Overview 

Under the Intelligent Robotics System Study (IRSS) contract, a generalized robotic 
control architecture has been developed for use with the ProtoFlight Manipulator Arm 
(PFMA) which resides at Marshall Space Flight Center (MSFC) in Huntsville, Alabama. 
Based upon the NASREM system design concept, the controller built for the PFMA 
provides localized position based force control, teleoperation and advanced path 
recording and playback capabilities. Various hand controllers can be used with the 
system in conjunction with a synthetic time delay capability to provide a realistic test 
bed for typical satellite servicing tasks. 

The PFMA has six computer controllable degrees of freedom (DOF) plus a seventh 
manually indexable DOF, making the manipulator a pseudo 7 DOF mechanism. 
Because the PFMA was not developed to operate in a gravity field, but rather in space, 
it is counter balanced at the shoulder, elbow and wrist and a spring counterbalance 
has been added near the wrist to provide additional support. Built with long slender 
intra-joint linkages, the PFMA has a workspace nearly 2 meters deep and possesses 
sufficient dexterity to perform numerous satellite servicing tasks. The manipulator is 
arranged in a shoulder-yaw, pitch, elbow-pitch, and wrist-pitch, yaw, roll configuration, 
with an indexable shoulder roll joint. 

Joints on the PFMA are driven via 7 pulse width modulated amplifiers(6 DOF + end 
effector). Resolvers and tachometers are used to measure relative joint positions and 
velocities. Motor drive currents are controlled via analog inputs to joint amplifier cards, 
and analog outputs are provided that feedback actual motor currents to the PFMA 
controller. Currently, there are two hand controllers for the PFMA: a compact rate 
(CAE) hand controller and a larger hand controller developed by Seargent 
Laboratories which has force feedback capabilities. A Lord Force/Torque Sensor 
mounted to the manipulator near its end effector is used to implement various force- 
based control schemes and eventually force reflection. 

Digital control of the PFMA is implemented using a variety of single board 
computers developed by Heurikon Corporation and other manufacturers. The digital 
hardware architecture is comprised of five Heurikon V2F processor cards (68020 
based) in a single 32 bit VME chassis connected to a second 24 bit VME chassis via a 
shared memory card. The first chassis (the computational chassis) performs all 
controls computations and operates the user interface. The second chassis contains 
all the electronics that interface directly to the PFMA and hand controllers. This 
chassis uses a Motorola MVME 104 processor card (68010 based) to perform all 


1 



system data acquisition. Because of its function, t.his second chassis is called the 
Input/Output (I/O) chassis. 

1.2 Scope of Phase IV Effort 

There were two major activities under the IRSS phase IV contract: 

1 ) Enhancement of the PFMA control system software functionality; 

2) Evaluation of operating modes via a teleoperation performance study. 
These activities are described in sections 2 and 3 respectively. 


2.0 Control System Software 

The following sections describe updates and enhancements made to the IRSS 
control software as part of the IRSS phase IV effort. Upgrades and modifications 
performed for this phase of the contract can be grouped into three principal areas. 
These categories are: Enhanced Operator Interface, Control System Upgrades and 
System Simulation Capabilities. Each of these areas may be further subdivided into a 
list of specific tasks which were performed to accomplish each. 

• Integration of the VPL data glove hand controller into the robot control 
system. 

• Integration of the Fluke 1031 touchscreens into the user interface. 

• Real-time kinematic simulation of the robotic manipulator using the Evans & 
Sutherland PS-390 interactive graphics computer. 

• Alternative PFMA joint reference generators were implemented. 

• Enhanced data recording capabilities were integrated into the PFMA robot 
controller. 

• Software/hardware related problems were identified and addressed. 

• Analysis, design and preliminary implementation of a limited bandwidth 
bidirectional communications link between a remote robotic servicer and a 
ground based user interface were performed. 

• Off-line system simulation capabilities were upgraded to support off-site 
update and debug of IRSS system software. 

2.1 Enhanced Operator Interface 

2.1.1 VPL Data Glove Integration. 

A spandex glove instrumented with a variety of sensors was integrated into the 
IRSS robot controller for use as a teleoperation input device. Due to the unique nature 
of this user input mechanism, it was anticipated that much could be learned as the 
glove was used to move the robot through its work environment performing various 
tasks. What makes the data glove so unique is that it has a much larger work volume 
than almost any mechanical hand controller, not to mention that it also possesses a 
very low inertia (a rather uncommon feature among hand controllers). This glove, 
developed and marketed by VPL Corporation, uses three separate types of sensors to 
determine a variety of conditions pertaining to the users hand position. A Polhemus 
electromagnetic emitter/detector pair uses a multiplexed magnetic field to determine 
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relative position and orientation of the glove withiq the hand controller work envelope. 
This is used to control gross robot position and orientation changes. Sixteen fiber 
optic flex sensors are mounted on the glove and are used to determine relative flexure 
of knuckles and the position of the thumb. This is used for controlling the end effector 
operations and virtual deadman switch (to perform ratcheting) for the robot. A final 
sensor set measures the relative distance between the tip of the index finger and the 
thumb (this sensor is not presently in use). 

The VPL data glove is tied to the IRSS control computer via a 9600 baud, RS-232 
serial link passing data to a MVME-410 card within the system VME I/O chassis (see 
the IRSS phase II final report 1 for a discussion of this chassis). Currently, six integer 
values representing glove Cartesian positions and orientation, and ten flex sensor 
values (2 for each finger and the thumb) are being returned to the I/O computer at a 30 
Hz rate. Orientations are input to the chassis in a yaw-pitch-roll format. Though while 
the I/O chassis is operating data acquisition from the data glove is constantly 
occurring, this data is only processed when data glove teleoperation is active. The 
resolution of incoming data is sufficient to permit the teleoperation of smooth robot 
slewing motions throughout the entire manipulator workspace. 

Using software developed for the data glove, a user defined virtual box is oriented 
around and located behind the glove's Polhemus emitter. With the glove inside this 
virtual box, when the thumb is curled tightly toward the palm, the PFMA can be 
teleoperated in a variety of modes including rate, position, and "hawk" (translations 
only mode). High and low gain settings for these various modes are correlated directly 
to middle and index finger positions as transduced by the glove's fiber optic flex 
sensors. High gain is achieved when both fingers are extended, low gain when only 
the index finger is extended. Flexing all fingers back commands the end effector jaws 
to open while curling all fingers toward the palm affects closure. Any glove gestures 
made when the glove is outside of the virtual box about the emitter are ignored by 
teleoperation algorithms. An interactive parameters menu permits the user to adjust 
various gain values and dictate thumb-deadman and finger-end effector open/close 
operational thresholds. Raw data coming from the data glove can be seen using 
hardware diagnostics provided in the IRSS software. 

Preliminary exercises with the data glove appear to demonstrate that it is well 
suited for the control of robotic manipulators like the PFMA. Although the RS-232 
serial link between the data glove and the I/O computer functioned very reliably, this 
input device was shown to have other rather troubling idio syn crasies with regards to 
the magnetic fields which it generates. The most promi ne nt had to do with the 
Polhemus emitter used to determine the glove 's orien tation and position, When this 
emitter was placed close to metal objects (e.g. metal desks and chairs) warping of its 
emission field caused the hand controller to behave abnormally along certain axis. In 


1 1ntelligent Robotic Systems Study (IRSS): Phase II Final Report, Martin Marietta, MCR-89-559, 
contract NAS8-36431, May 1990. 
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addition, it was determined that operating the VPL glove in close proximity to the 
system I/O chassis appears to induce electrical noise onto the VME backplane causing 
failure of system bus read and write cycles; a very serious problem. To optimize the 
operation of the data glove more effort is needed to calibrate operational axial gains 
for use in all modes that the controller can function in. This is necessary to guarantee 
the safety of the PFMA and fragile task panels. 

A preliminary application of neural networks to help classify gestures and 
commands was performed. Although algorithms for their use were not integrated into 
the IRSS controller, a small amount of data was collected and used with a simple 
network. The results were very promising. Rapid convergence during training and 
distinctive recognition of gestures was achieved for all gestures in each of four data 
sets collected. 

2.1.2 Fluke 1031 Touch Screen Integration 

The new human-computer (HCI) system and associated screens allow the operator 
to easily operate the PFMA. The hierarchy for the new HCI system is shown in Figure 
2-1. The new HCI design provides an easy to use, hierarchical operator interface. In 
the Help, Demo, and Test hierarchies, the operator is provided with prompts to guide 
him/her through the hierarchies. These hierarchies provide the operator with two 
methods of command entry — touchscreen and keyboard input. The Program hierarchy 
screens do not provide prompts to guide the operator though the hierarchy and 
provides primarily a keyboard entry interface. The Program hierarchy provides less 
guidance to accommodate the skill level of experienced programmers who will be 
using this hierarchy of screens. 

This HCI design was implemented by adding two Fluke 1031 touch terminal 
monitors to the previous IRSS operator interface. Using graphics characters and 
resident highlighting and font capabilities present in these terminals a simplified user 
interface was added to the IRSS robot control system. Using these terminals many 
macro operations requiring many keystrokes from the engineering console were 
consolidated under a single touch key. 

The two touch terminals are physically connected to the Heurikon real-time control 
computer via 9600 baud RS-232 serial Jinks (through the XYcom quad serial port 
card). The program which operates the user interface routinely queries each of the 
touch screens to determine if the user has selected an operation through either touch 
or by having typed a letter at the touch screen keyboard. Operations coming from the 
touch screens behave just as they would if they came through the engineering console 
terminal. Functionality is divided between the two touch screens with the first 
operating as the sole operations command screen and the second providing a system 
monitor and display function. 
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Figure 2-1. Touch Screen Hierarchy 

The system command touch terminal provides four primary menus used to control 
the PFMA i n a numb er of modes and using variou s ha nd contro ll ers. Additional 
menus are pro vided that allow the user to move from menu to menu. Stubs for future 
operational menu integration have also been provided. Typically, operational menus 
provide selection of impedance parameters from a choice of four options, selection of 
hand controller operational modes and gains and execution of one of up to four 
autonomous path primitives. Touch menus for both the VPL data glove and the CAE 
hand controller are provided. One of the touch screens is designed specifically for the 
analog like calibration of impedance parameters using either the CAE or the VPL data 
glove hand controller. A topological map of the touch screen menu tree is provided via 
a touch key common to almost every screen. 

The system monitor and display touch screen allows the user direct visibility of 
system variables associated with each of the functional layers of the IRSS control 
system (for a definition of these layers see the IRSS phase II final report). These 
displays are updated at near real time rates to provide accurate internal system 
information. Help information is available at the touch of a key. This real-time 
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monitoring function can also be disabled via an active touch key. With this monitoring 
and display function disabled operator input bandwidth via the engineers console is 
improved. 

The entire Fluke touch screen interface is built upon a standard touch screen 
library designed to operate the touch terminals and design custom menus with a 
minimum of effort. Standard utilities for the display of real time menus were adapted 
for use with these terminals reducing the overall amount of code that had to be 
developed. This interface library should be portable to other hardware systems (e.g. 
PCs or VAXs) so that almost any system with a C compiler, the support library and a 
serial port can function as a host. This support library provides primitive functions such 
as complete touch key definitions, line drawing and text processing. 

2.1.3 Real-time Kinematic Simulation of the PFMA 

With a variety of mechanical limitations which the PFMA has, the robot controller 
can sometimes appears to be malfunctioning. These physical limitations include 
stiction in both the joint motor drive trains and the spring counterbalance supporting 
the robot's wrist, as well as limitations on the maximum amount of torque that the 
PFMA's joints can generate. This manifests itself in teleoperation, for instance, when 
commanded motions from a hand controller appear to have to no effect on the robots 
position. To enhance the operator interface, help understand some of the physical 
limitations of the mechanical system and provide the capabilities to playback 
autonomous path primitives before they are executed, a graphical kinematic 
simulation of the PFMA was developed. In this simulation a wire-frame image of the 
PFMA is displayed indicating the commanded positions of each of the robot's joints 
and end effector. This graphics display can be scaled, and the observation point from 
which the simulation is viewed can be modified using several dials provided with the 
graphics computer, an Evans & Sutherland PS-390 graphics workstation. 

This simulation is driven via an RS-232 link connecting the Heurikon computational 
chassis (using the XYcom quad serial port card) to the serial input port of the E&S PS- 
390. After the preliminary wire frame model of the PFMA is downloaded to the PS- 
390, the computer is periodically passed the commanded joint angles of the PFMA. 
This provides real-time animation of the robot's motion. This simulation can be driven 
using either teleoperation (CAE or Data Glove hand controller) or autonomous path 
primitives recorded by the user. The display also reflects changes in the robot's 
commanded position as dictated by forces applied to the end effector that are 
processed by the impedance control subsystem. This functionality provides for 
operation for the PFMA’s digital control system when either the analog-l/O system is in 
a state of maintenance and repair or when "safe" debugging of new controller 
algorithms is necessary. The ability to execute autonomous task primitive motions in 
this manner ensures that when these segments are executed using the PFMA they will 
complete their intended operation. 


7 


2.2 Control System Upgrades 


2.2.1 Robot Servo Command Input Upgrades 

During the IRSS phase II effort joint performance data was collected for the PFMA 
and analyzed in an effort to improve joint disturbance response characteristics. The 
difficulty with which this was accomplished illustrated the need to provide utilities 
within the IRSS robot controller to manually inject various wave forms directly into the 
joint controllers of the robot for the purpose of analysis and diagnostics. During IRSS 
phase IV this capability was implemented providing the ability to inject sine and 
square wave functions into the forward path of each of the robot's independent joint 
controllers. Using this facility and a user defined configuration menu, the amplitude, 
frequency and relative phase of these references for each of the robot's joints is user 
configurable. 

2.2.2 Data Recording and Analysis Upgrades 

To perform human factors experiments during phase IV of the I RSS co ntract the 
addition of a data recording capability was implemented and integrat ed into the robot 
controller. Because important feedback information is distributed throughout the robot 
control system, fundamental components of the recording facilities must function as a 
unit across multiple processors each with the ability to perform independent data 
recording. To meet these requirements, a set of software modules was built around a 
common data structure and set of low level software utilities, each capable of 
functioning independently on an isolated processing node (Figure 2-2). 



Figure 2-2. Distributed Data Recording and Display 
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The data recording capability has several primary components; raw recording, raw 
display, transfer to mass storage and graphics display for analysis purposes. The raw 
data recording facility permits the user to select data to be recorded, its recording rate, 
its recording mode and the amount of memory used to record the particular data item. 

A selection from over 1 5 different data items is currently provided. Start, stop and reset 
of data recording is controlled from the main engineering console terminal. Using the 
data display utility the user can scroll through recorded data, stopping it as necessary 
to examine individual items. To quickly examine the recorded data a graphics support 
system is provided that scales and passed the recorded data (via RS-232) to the 
Evans & Sutherland PS-390 graphics computer. On the PS-390 display a plot is 
produced complete with grid marks, title, axis labels and a legend in full color. Up to 
five overlaid plots can be produced making comparison of system performances fast 
and efficient. The user has complete control over nearly all aspects of the plot 
produced. Permanent storage of recorded data is accomplished by translating binary 
values to ASCII and then saving them in user defined files on the Heurikon hard disk. 
From this disk they can be transferred to other remote systems using either streaming 
tape or floppy. Currently, there is approximately 400K of memory available for use 
with data recording. Filling all of this memory with data produces an output disk file 
which when saved is nearly 1 megabyte in length. 

2.2.3 Software and Hardware Problems Addressed 

In the course of performing phase IV work a variety of software problems were 
identified and corrected. These included problems associated with sometimes 
unreliable power-up and down of the system and structure of the system world model. 
To provide support for the serial interface to the VPL data glove an additional MVME- 
410 quad serial card was added to the I/O chassis. Due to the large amount of serial 
data processed by the I/O computer, this addition precipitated the need to convert I/O 
chassis hardware and software operating the serial interface from a poling method to 
an interrupt driven scheme. This upgrade not only incorporated the data glove hand 
controller but also improved the reliability and speed of the Lord Force/Torque sensor 
interface. Documentation of existing and newly developed system software was 
improved to help clarify system operations and data flows. 

Despite the fact gains in system reliability have been made during the phase IV 
effort, there do remain several significant problems with the system. The first of these 
problems has to do with the interaction of the system I/O chassis and the Polhemus 
emitter functioning with the VPL data glove. Because of electrical noise induced by 
fluctuating magnetic fields coming from the emitter, the I/O chassis will occasionally 
receive spurious signals while attempting to access the VME bus. This typically results 
in a failure indication, "crashing" the system. Improved shielding of the I/O chassis 
backplane would be the most obvious solution to this problem. A second problem is 
that one of the Heurikon V2F real-time computational processor cards occasionally 
fails indicating a memory parity error. The suspected cause in this case is that the 
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VRTX kernel software which performs multi- tasking on the board was not saving and 
restoring floating point coprocessor registers when appropriate. A simple upgrade of 
VRTX software from version V20 to V32 should eliminate this difficulty. With these 
problems data collection during the experimental phase of the phase IV effort was 
hampered, requiring an excessive amount of individual data file processing to occur 
before the information could be analyzed. 

2.3 System Simulation Capabilities 

2.3.1 Simulation of Limited Bandwidth Communications Link 

The design and preliminary implementation of a limited bandwidth, bidirectional 
data link was accomplished during phase IV of the IRSS contract. The objective of this 
task was to examine the effects of extremely limited data communications rates upon 
teleoperation, system safety and system controllability of a robotic system whose 
operator and user interface are separated from the real-time control system and robot. 
In total, it was anticipated that this system would emulate an Earth base control station 
for an on-orbit remote manipulator such as the Orbital Maneuvering Vehicle (OMV), 
the Flight Robotic Servicer (FTS), or the Satellite Servicing System. 

The bandwidths selected for this system were roughly equivalent to that which 
were expected for the proposed OMV. The uplink data rate from the control station is 
designed to be user configurable ranging from 230 to 810 bits/second. Uplink data 
packet sizes range from 23-27 bits/packet with transmission rates of between 10 and 
30 packets/second as selected by the user. In the preliminary design, the downlink 
bandwidth was significantly greater than that of the uplink, ranging from 2300 to 6900 
bits/second with packet sizes of 230 bits/packet at rates from between 1 0 to 30 
packets/second. Time delays for packets are user configurable ranging from 0 to 5 
seconds each direction. 

A strategic method for optimizing system data integrity between the various 
components of the distributed control system was developed. The goals of the 
methodology developed was to optimize the data flow through the constrained link by 
eliminating the transfer of redundant information and providing a criticality prioritized 
scheduling framework for the transmission of system data. On the uplink side, the 
ability to perform robust teleoperation of the manipulator across the data link was 
desired. A block diagram detailing the organization of this system is shown in Figure 
2-3. A description of the major components found in the system is provided below. 

2. 3. 1.1 Delay Blocks 

These blocks emulate a realistic time delay anticipated between the Earth based 
ground station and a remote robotic servicer. Delays are user selectable and can 
range from 0 to 5 seconds. 
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Ground Based User Interface Remote Servo Manipulator 


Figure 2-3. Reduced Bandwidth Communications Link Simulation 

2.3. 1.2 World Models 

Within this system there are three separate world model entities. The first is the 
Remote Robot World Model. This world model is the true informational data base from 
which the real-time operations of the robot occurs. The system real-time 
computational and I/O processors update this world model causing tasking data to flow 
in the remote controller. The second world model is the Downlink World Model 
maintained by the monitor and compare function on the remote robot control system. 
The monitor and compare subsystem compares data within the Remote Robot World 
Model (the true world model) to this local model and generates a list of discrepancies 
between the two. The Downlink World Model is intended to be a duplicate of what is 
the third world model, namely, the Ground Based World Model. The Ground Based 
World Model is the only record that the remote operator interface has for determining 
the operational state of the remote manipulator servicing system. 

2. 3. 1.3 Remote Monitor & Compare 

The remote monitor and compare function is used to help the Ground Base World 
Model reflect as accurately as possible the state of the Remote Robot World Model. 
This function is constantly comparing its local world model with the Remote Robot 
World Model identifying items which differ. Differences between the two indicate that 
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the Ground Based World Model does not accuratejy reflect the true state of the Remote 
Robot World Model and so transmission of this discrepancy is necessary. This 
function helps ensure that packet bandwidth is not wasted transferring data to the 
ground station that currently reflects the existing state of the remote world model. 
Output from this function is a list of world model data items requiring update on the 
Ground Based World Model. This list of data items in need of transfer is maintained 
and managed in a queue. 

2. 3. 1.4 Packet Transfer Scheduler 

The packet transfer scheduler is the mechanism which at the downlink 
transmission rate determines the contents of each successive downlink packet. 
Fundamental definition of the data scheduled for downlink transfer is determined by 
the user (see the discussion of the User Interface in this section). Normally, for any 
given transmission frame the user will have identified the particular piece of data to be 
transferred via the downlink. In "spare frames" where this is not explicitly dictated, the 
Packet Transfer Scheduler goes to the head of the Change Packet Queue and 
removes the highest priority item, preparing it for transfer. With each world model data 
item possessing its own relative criticality level (thus providing a rough measure of its 
importance), transfer priority of items in the Change Packet Queue can be assessed. 

In the current implementation the position of a data item in the downlink transmission 
queue is determined by a combination of how long the item has been in the transfer 
queue, and its relative criticality. In some cases a data item of low criticality may be 
transferred ahead of another data item which possess a higher criticality because it 
has been in the transfer queue significantly longer. This queueing approach 
guarantees that data items which have relatively low criticality do not starve (e.g. they 
are never transferred), but that data items which have greater significance are more 
rapidly updated as they change. 

2. 3. 1.5 User Interface 

Through the user interface, configuration of the remote link simulation is controlled. 
For the definition of the uplink a menu is used to set the packet transfer rate, the packet 
size (in bits), the time delay and encoding method used to transfer remote control data. 
Operation of the remote system si mulato r can also be ena ble d and disable from this 
menu. A second menu to control the downlink permits the user to set the downlink 
packet rate and to explicitly configure the transmission rate of the following world 
model data items: 

• Robot Commanded Position 

• Commanded Joint Positions 

• Joint Voltage Commands & Motor Tachometer Impedance Offset 

• End Effector Forces 

• World Forces 


• Actual Joint Positions. 

• System State Variables 

• General System Data that is Changing 

As the remote link simulation is enabled and disabled, communications tools used 
with the world models redirect access appropriately between the Ground Base World 
Model and the Remote Robot World Model. The fact that this is occurring remains 
totally transparent to the modules accessing world model data items. This greatly 
enhances the ability to debug the system allowing the user to browse multiple world 
models as needed. 

2.3. 1.6 World Model Change Request Queue 

On the ground based user interface side of the remote link, commands and change 
requests coming from the user interface are queued before they can be transferred to 
the remote system. Requests wait in this queue until they are packetized and sent. 
Once a data item has been packetized and is passing through the delay subsystem, 
the new state of this world model data item is reflected in the Ground Based World 
Model. On the remote manipulator side of the link, whenever a packet is received both 
the Downlink and the Remote Robot World Model are updated to reflect the new state 
of the system. If the particular data item updated is found in the downlink packet 
queue (ready for downlink transmission) it is then removed. 

2.3. 1.7 Uplink Priority Scheduler 

The Uplink Priority Scheduler is used to determine the relative sequence of 
packets which are transferred from the user interface/ground station simulator to the 
remote manipulator. A simple priority based scheduling algorithm is used, preempting 
any stream of direct remote command packets with any other command type being 
issued from the user interface. Since typically the number of commands coming from 
the user interface directed to the real-time controller is small, a First In, First Out (FIFO) 
queue is used to hold pending commands. When no pending user interface uplink 
control commands are available for transmission, the uplink continues to transfer 
remote direct command packets according to the mode of operation (Cartesian servo, 
joint servo or teleoperation) of the ground based system. 

2.3.1 .8 Packetization/Depacketlzation 

For the data being transferred to and from the remote manipulator packetization 
and depacketization is necessary. Within the system, uplink packaging is much more 
sophisticated that that for downlink data due to the narrower bandwidth of the uplink 
side. Before transmission, command information must be compressed if robust 
teleoperation is to be possible. A discussion of these compression techniques is 
provided in the following paragraphs. 

There are currently three specialized modes of transfer which are used to directly 
control the motion of the remote manipulator. These three modes are the transmission 
of an absolute robot commanded position, the transmission of a Cartesian delta 



(typically associated with a hand controller), and the transmission of absolute robot 
joint commands. In each case it is taken for granted that six degrees of freedom (DOF) 
of information is to be transferred. In each case real numbers are normalized and 
scaled across some metric of the robot world environment (maximum positions for 
Cartesian workspace or joint angle or maximum delta respectively). This reduces the 
storage and transmission requirements for the data without unduly diminishing its 
accuracy. 

For the direct remote command of the manipulator (this implies direct transmission 
of Cartesian or joint absolute positions or deltas) there are currently four encoding 
methods proposed for data transfer. The first and most simplistic of these transfer 
schemes is the Standard Transfer packet method (Figure 2-4). This type of packet is 
comprised of a more bit (indicating that the packet is just one more in a continuous 
stream), a 2 bit DOF selector field followed by 2 individual data fields containing the 
command data for the robot. When the packet type is a robot Cartesian reference 
command the data fields contain normalized integer representations of either X and Y, 
Z and Yaw, or Pitch and Roll Cartesian commanded positions. The degree of freedom 
selector field indicates which of these coordinate pairs is present in a given packet. As 
the length of these packets range from 23 to 27 bits, additional bits are successively 
added to each of the data fields in a left first, then right, then left again fashion. When 
uplink packets are sized at 23 bits each field is 1 0 bits in length. When the packet size 
is 27 bits each data field contains 12 bits. When hand controller delta commands are 
being processed the data fields hold X & Y deltas, Z delta and Q1 of a quaternion, or 
Q2 and Q3 of a quaternion (Note that given Q1 , Q2, and Q3, Q4 can be determined by 
normalizing the quaternion vector; this is safe for delta transmissions but is 
undesirable for transmission of commanded Cartesian orientations because of 
relatively large errors associated with discretization). In the case of the joint reference 
command packet, base and shoulder, elbow and first wrist, or second wrist and third 
wrist commanded joint position data is scaled and encoded. 


1 

2 bits 

10-12 bits 

10-12 bits 

T 

t 

t 

Data Field 

t 

Data Reid 


| DOF Select 
More Bit 

Figure 2-4. Uplink Data Compression: Standard Transfer Packet 

The second encoding scheme used for the direct remote command uplink was the 
high resolution sliding scale methodology (Figure 2-5). This technique provides three 
data fields in each packet as opposed to two (as in the Standard Transfer packet). In 
this case, only 1 bit to select the DOF is needed (assuming uplink data will never have 
more than 6 degrees of freedom). This packet type differs from the Standard Transfer 
packet in that in addition to the third data field a multiplier field also exists. This 




multiplier dictates the number of left bit^shifts that the data found in the 3 data fields 
must be subjected to before they are embedded into output data. In 23 bit packets this 
is a 3 bit field (providing an 8X multiplier), and in all larger packets it is 4 bits (providing 
a 16X multiplier). As additional bits are successively added to the packet size they are 
evenly distributed among the data fields left to right. When Cartesian robot reference 
command packets are provided they contain either X,Y and Z, or yaw, pitch and yaw 
commanded position components within the data fields. Similarly, hand controller 
delta packets will contain either X,Y and Z scaled deltas or Q1 ,Q2 and Q3 scaled 
quaternion values. Finally, joint reference command packets will provide either scaled 
base yaw, shoulder pitch and elbow pitch commands or scaled first wrist, second wrist 
and third wrist commands. Note that using this multiplier a higher resolution of robot 
commands can be maintained than is possible with Standard Transfer packets. 
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Figure 2-5. Uplink Data Compression: High Resolution Sliding Scale Packet 

The third encoding method, referred to as the High Speed Sliding Scale encryption 
scheme, is comprised of one data packet with data for all six DOF of the robot (Figure 
2-6). Each data field is either 3 or 4 bit of differential data for each DOF (note that 
because all are differential transmissions this is different from other packet types). A 
scale field of 2 bits provides a shift multiplier to be applied to each of these data fields 
before they are combined with existing data to form an output command. Commanded 
Cartesian reference delta packets contain the X, Y, Z and yaw, pitch, yaw differentials 
successively. Hand controller differentials are transferred as X, Y, and Z deltas 
followed by Q1, Q2, Q3 quaternion differential components. Commanded joint angle 
data ordering follows the robot’s sequential joint structure from shoulder yaw to the 
third wrist. 
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The fourth direct command encoding scheme js called the Modified Sliding Scale 
(Figure 2-7). This method is very similar to the High Resolution Sliding Scale method 
and the Standard Transfer technique. This method contains two separate data fields 
each with a separate DOF selector field. The encoding method of a shift multiplier 
followed by a scaled integer value for the particular degree of freedom is synonymous 
with the technique use in the High Resolution Sliding Scale method. The contents of 
the data packets (as normalized integer values and their order) are also equivalent. 

An advantage offered by this encoding method is that information for two independent 
degrees of freedom can b e transferred in one packet, each with its own shift multiplier 
which can optimize the p ositio n of the data in th e da ta field in the packet. A technique 
such as is in floating point math where an implied bit is found to the left of the mantissa 
could be applied here to increase the information transferred in each packet. 
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Figure 2-7. Uplink Data Compression: Modified Sliding Scale Packet 

Each of these encoding schemes has its relative advantages and disadvantages. 
The Standard Transfer packet is simple to encode and decode, but requires a 
minimum of three packets to form a complete 6 DOF command. Another disadvantage 
of the standard transfer scheme is that the smaller the size of the data packet, the less 
resolution the data in the packet has. With only two packets the High Resolution 
Sliding Scale packet can form a complete command (a 33% improvement over the 
Standard Transfer), however, significant bits may be lost when actual commanded 
positions are being transferr ed requiring the system to wait for transmission of lower 
order bits before a truly complete command is received. Although in the case of 
absolute position commands this can pose a problem, transmission of delta in this 
fashion can be very efficient (because differential information can be scaled to 
minimize use of the upper bits in a value). The High Speed Sliding Scale algorithm 
can encode a complete command in only 1 packet but can only be used in a 
differential transmission mode, not in an absolute command mode. Also, because 
each data field is so small, there is always a tradeoff between bandwidth of motion and 
quantization errors. The Modified Sliding Scale method combines the advantages of 
the High Resolution Sliding Scale with the flexibility of being able to select 
independent degrees of freedom concurrently. Its disadvantage is that it can have a 
lower bandwidth than the Standard Transfer scheme if all robot degrees of freedom 
are changing rapidly. 
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Downlink packaging is much simpler, with each packet containing 230 bits. These 
bits are divided into 18 individual fields (see Figure 2-8). The right most components 
are comprised of fourteen 16 bit data fields. These data fields are filled with raw data 
dumped into them in a left field first fashion. No specialized compression goes on. 
This permits the transfer of up to seven 32 bit reals, fourteen 1 6 bit integers or twenty- 
eight characters per packet. The left most bit in the packet (the more bit) indicates 
whether the packet is a continuation in a stream of packets, and the power-up bit 
indicates whether the remote manipulator is actively powered up. An error bit is set to 
indicate that the robot is currently in an error state and a mode field is used to indicate 
information about the current configuration and state of the uplink. Predefined packet 
sequencing and synchronization is assumed on both ends of the data link. This 
reduces overhead and increases system bandwidth by permitting multiplexing of data 
through the manipulator to ground station link. 
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Figure 2-8. Downlink Data Compression Packet Format 

To fully implement the narrow bandwidth remote link simulation system described 
here additional processing capability had to be added to the Heurikon/IRSS 
computational chassis. Selected was a single Heurikon V2F processor card much like 
those already performing real-time calculations in the computational chassis. The only 
difference between this processor card and others in the same chassis is that the new 
processor card has 4 megabytes of on-board memory as opposed to 1 megabyte. In a 
hardware/software interaction sense, Figure 2-9 contrasts the phase II world model to 
user interface communications method to that implied in this discussion. As can be 
seen, the additional processing capability simply acts as the link between the user 
interface and the real-time robot control/servo system. 
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Figure 2-9. Data Flow Comparison for Phases II and IV 

In summary, the system designed for the simulation and control of a remote 
manipulator provides a realistic and practical approach to overcoming problems 
associated with a limited bandwidth link with variable time delay. Although every 
aspect of the proposed design was not fully realized in the phase IV effort, a 
foundation has been established upon which further enhancements can build. It is 
expected that results coming from this implementation and the study of such, could 
have significant impact upon the assessment and feasibility of performing many 
satellite servicing tasks from a ground base control station. With the flexibility that this 
system provides in terms of data rates and packet sizes, a variety of comparison and 
performance studies could be performed that evaluate packet encoding 
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methodologies versus user response bandwidth. In the long term, methodologies for 
swapping of encryption techniques as a function of the operational motion may need 
to be considered. 

2.3.2 Off-line Simulation Capabilities 

To provide the capability to develop software for the IRSS system independently 
from the PFMA and related hardware, a non-real time system simulator has been 
gradually developed and upgraded through phase II and phase IV of this contract. 

This simulator permits an IBM-PC to be used as a hardware platform on which real- 
time software can be debugged, evaluated, modified and enhanced. Using this 
development environment approximately 85% of the software modules which are used 
in the real-time control of the PFMA are linked directly (without source code changes) 
to simulation software utilities. Operating on a PC, a large variety of tools are available 
for debug, configuration management and graphics. Because the simulation is non- 
real time, relative synchronization of tasks and system data flows can be examined at 
various levels of detail. , 

At the heart of this simulation is software that emulates the VRTX multi-tasking 
kernel found on each of the Heurikon V2F real-time computational processor cards. 
This kernel maintains records of which system tasks are active and executes them 
according to their relative priority and availability. The full IRSS system user interface 
executes concurrently with the real-time simulation allowing the user to monitor system 
parameters and inject new commands and disturbances into the system. A standard 
plant joint model is used to emulate independent joint dynamics for all six degrees of 
freedom of the robot. With the modular nature of these plant models as well as the 
various joint controllers this system can be used to simulate robots other than the 
PFMA and design joint controllers for such. 

Without hand controllers, touch screens or the PS-390 interfaced to the PC, 
operation of the simulation via teleoperation, touch, and graphics is not currently 
provided although these capabilities could be added. Due largely to the fact that 
portability of system software has recently been considered, porting of this simulation 
to a Unix based workstation or mainframe computer could be performed to increase 
the system's speed of execution and its capabilities. Currently, continued 
enhancements of the IRSS control system are being performed using the PC. 
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3.0 Teleoperation Performance Study 


3.1 Introduction 

This study evaluated three issues related to remote control of robotic systems. 

These three issues were the control reference frame, impedance level, and amount of 
time delay present in the system. The effect of different levels of these parameters on 
operator performance was determined. 

3.1.1 Reference Frame 

Reference frame refers to the location and orientation of the axis of control. The 
reference frame can be fixed in space or can be located with respect to a moving point 
(object) in space. An example of a reference frame fixed in space would be a 
reference frame coincident with a fixed camera or fixed portion of the robot, for 
example the shoulder. With a fixed reference frame, the a xes woul d re main fixed 
regardless of the orientation of the robot gripper. For example, +Z would always point 
up from the base of the robot (even if the gripper was rolled 90 -degrees such that it 
was looking upside down). An example of a reference frame not fixed in space would 
be one coincident with the gripper or a gripper mounted camera on the manipulator. 

In this case, the reference frame changes to correspond to the orientation of the 
gripper (or gripper mounted camera). For example, +Z would always be up from the 
gripper (as opposed to up in fixed space). The sele ction of th e locatio n of the 
reference frame and whether or not the reference frame is fixed in free space or fixed 
with respect to a moving object is based on many considerations such as the type of 
task being performed, size of the workspace, and feedback being provided to the 
operator. 

3.1 .2 Impedance 

Impedance refers to the apparent stiffness and damping of the manipulator at the 
end effector. Impedance control laws allow compliance to be programmed, reducing 
the forces resulting from environmental contact In Cartesian space, end effector 
forces measured by the wrist-mounted force/torque sensor can be utilized to modify 
the effective impedance of the end effector using position-based impedance control. 

A stiff manipulator exerts large forces on the environment during contact in response to 
small commanded motions. This can result in physical damage to the contacting parts. 
A soft manipulator yields when in contact, resulting in smaller interaction forces. If the 
impedance control is too soft however, the task may be difficult or impossible. 
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3.1.3 Time D«iay 

One of the problems that arise in remote control of robotic systems is the 
introduction of time delays in the transmission of commands from the operator to the 
robot and feedback from the robot to the operator. Delays on the order of 2 to 4 
seconds are reasonable for ground control of space robotic systems. 2 Studies have 
shown that operator performance is affected negatively by the introduction of time 
delay for such tasks as fine precision peg-in-the-hole 3 and pursuit tracking, 4 and that 
operators adopt a move-and-wait strategy. 5 

3.2 Research Methodology 

This study was conducted at the Marshall Space Flight Center (MSFC) in the 
Robotics Laboratory. The operators controlled the Proto-Flight Manipulator Arm 
(PFMA) to perform a simulated Orbital Replacement Unit (ORU) replacement task. 

3.2.1 Apparatus 

Figures 3-1 and 3-2 show the layout of the various components used in this study. 
Two task panels were used and each had an ORU receptacle. Contact switches were 
mounted on the rear face of the receptacles to detect when the ORU was completely 
inserted. Feedback of successful insertion of the ORU was provided to the operators 
by the illumination of an Light Emitting Diode (LED) located above each ORU 
receptacle. Two infrared photo-interruptor sensors were placed on standoffs of 
approximately 1 .5 inches in front of each panel to indicate when the ORU was near the 
front face of the panel and receptacle (Figure 3-3). This sensor information was used 
to signal the transition between the slewing and fine alignment task segments. For 
example, Task Panel (TP) #1 Insertion subtask started when the ORU was sensed by 
the sensor on the standoff and ended when the sensor on the rear face indicated that 
the ORU was completely inserted. 


2 Buzan, F. T. and Sheridan, T, B. (1989, November). A model-based predictive operator aid for 
telemanipulators with time delay. In Proceedings of the 1989 IEEE International Conference on Systems, 
Man, and Cybernetics. November 14-17, 1989. Cambridge, MA. NY: IEEE. 

3 Hannaford, B. and Kim, W. S. (19, November). Force reflection, shared control, and time delay In 
telemanipulation. In Proceedings of the 1989 IEEE International Conference on Systems, Man, and 
Cybernetics. November 14-17, 1989. Cambridge, M A. NY: IEEE. 

4 McCormack, L. B., Detroit, M. J„ and See, D. N. (1987, May). Improving pilot-vehicle integration 
using cockpit display dynamics. In Proceedings of the IEEE 1987 National Aerospace and Electronics 
Conference NAECON. May 18-22, 1987. Vol. 2. NY: IEEE. 

5 Ferrell, W. R. (1965). Remote manipulation with transmission delay. IEEE Transactions on Human 
Factors in Electronics, HFE-6 (pp. 24-32). Cambridge, MA: IEEE. 
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Figure 3-1. Top View of Worksite. 

The operators controlled the PFMA via a 6-degree of freedom (DOF), rate CAE 
hand controller located at the workstation (Figure 3-4). The CAE hand controller fits in 
the palm of the operator's hand and has an activation switch located at the index 
finger, which must be fully depressed to issues commands to the PFMA. The 
workstation has three-bays and is configured like the console currently envisioned for 
the Orbital Maneuvering Vehicle (OMV). Partitions were placed between the operator 
and PFMA such that the operator could not see the worksite directly. Two black and 
white camera views of the worksite were presented on the console monitors. 
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Figure 3-2. Front View of Worksite. 


3.2.2 Task 

The task simulated the major elements of an ORU replacement task. This task 
required the subjects to remove the ORU from TP #1 receptacle, slew to TP #2, insert it 
into TP #2 receptacle, remove it from TP #2 receptacle, slew back to TP #1 , and insert 
it into TP #1 receptacle (refer to Figures 3-1 and 3-2). The subjects received feedback 
when the ORU was inserted fully via an LED located above each ORU receptacle. The 
task contained two distinct components— large, non-environmental contact slewing 
motions, and fine alignment and positioning motions. 

3.2.3 Subjects 

Twelve students from the University of Alabama at Huntsville participated in this 
study. Six of the subjects were male and 6 were female. The subjects were paid 
$10.00 per hour for their participation. None of the subjects were familiar with the 
experimental task, hand controller, or manipulator used in this study. 
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Figure 3-4. Photograph of Subject at Operator Workstation. 


3.2.4 Procedure 

The subjects attended a training session during which they were informed of the 
experimental setup, including information on the operation of the PFMA, hand 
controller, and location of cameras. The subjects were informed of their rights as study 
participants, signed an Informed Consent Form (located in Appendix A), and filled out 
a background questionnaire (located in Appendix B). The task was explained to the 
subjects and demonstrated by the test monitor. The operators were trained on the 
system by performing six training runs that were performed in the following order: 
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• End Effector Reference Frame, Stiff Impedance, No Time Delay 

• End Effector Reference Frame, Soft Impedance, No Time Delay 

• World Reference Frame, Stiff Impedance, No Time Delay 

• End Effector Reference Frame, Stiff Impedance, 1 Second Time Delay 

• End Effector Reference Frame, Stiff Impedance, 2 Seconds Time Delay 

The first training run was viewed as the baseline condition and was run twice. In 
the four remaining runs, one parameter changed from the baseline. The operators 
were informed of the changes between training runs so that they could better 
understand the differences observed in system performance. 

In addition to the two camera views provided on the control console, the subjects 
were allowed to view the task directly and were provided with a graphic simulation of 
the manipulator during the training session. The subjects were aware that the direct 
view of the task and graphic display would not be available during the experiment. The 
participants were told that it was more important to concentrate on performing the task 
well, as opposed to performing it fast. 

At the beginning of each set of data collection runs, the subjects were given a 
practice run with the baseline condition. Prior to each data collection run, the 
participants were told the reference frame condition for that run, but were not told the 
impedance or time delay conditions. At the completion of each experimental run, the 
participants provided the test monitor with a Cooper-Harper (C-H) 6 rating of the task 
difficulty (rating scale is located in Appendix C). After the completion of all 12 
experimental runs, the subjects completed a post-test questionnaire (located in 
Appendix D). Experimental runs that could not be completed within 10 minutes were 
stopped and were recorded as incomplete. Complete task segments from incomplete 
(discontinued) runs were included in the analysis. 

3.2.5 Experimental Design 

A within-subjects design was used, such that all of the subjects experienced all of 
the experimental conditions. A fully counterbalanced presentation order was used, 
meaning that no two subjects experienced the experimental conditions in the same 
order. 

3. 2. 5.1 Independent Variables 

Reference frame control mode, time delay, and impedance level were evaluated in 
this experiment. 


6 Wierwille, W. W. and Casali, J. G. (1983). A validated rating scale for global mental workload 
measurement applications. In Proceedings of the Human Factors Society 27th Annual Meeting. 
October 10-14, 1983. Norfolk, VA. Vol. 1, pp 129-133. Santa Monica, CA: HFS. 
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Reference Frame Control Mode —. This study svaluated two reference frame 
control modes— a World and an End Effector frame of reference. The world reference 
frame was located at the point of intersection of the PFMA shoulder pitch and shoulder 
yaw. This control mode mapped hand controller motion relative to a fixed coordinate 
system in space, regardless of the orientation of the end effector. The end effector 
reference frame was located at the center of the gripper and its coordinate system 
varied in space relative to the orientation of the end effector. For example, in the World 
Reference Frame condition, the +X direction was always pointing towards Task Panel 
#1 . In the End Effector Reference Frame, +X was always pointing out of the end of the 
gripper, regardless of the orientation of the gripper. When the end effector and world 
frames were aligned, such as when the ORU was inserted into Task Panel #1 
receptacle, these two control modes were the same. However, when the two frames 
were not aligned, such as when inserting the ORU into Task Panel #2 receptacle, the 
control modes were different. 

Impedance - Two levels of impedance were evaluated. The first level was 
referred to as "stiff and consisted of the following stiffness and damping values: 
Stiffness: kx = 2 ky = 2 kz = 2 kox.koy.koz = 20 

Damping: bx = 10 by = 10 bz = 10 box.boy.boz =100 

The second level was referred to as "soft" and consisted of the following stiffness 
and damping values: 

Stiffness: kx = 2 ky = 10 kz = 2 kox.koy.koz = 20 

Damping: bx = 10 by = 10 bz = 10 box.boy.boz “ ^00 

The parameter and value that differed between the two conditions is noted in bold. 
Time Delay - The three levels of time delay that were evaluated were no time 
delay, 1 second time delay, and 2 seconds of time delay. 

3. 2. 5. 2 Dependent Variables 

Several objective performance measures were collected, including the amount of 
time required to perform the task, successfulness of inserting the ORU in the 
receptacle properly, and forces and torques at the end effector during insertion. The 
ORU insertion success was based on whether or not the subjects fully inserted the 
ORU into the ORU receptacle. This was determined by sensing the contact switch 
mounted on the back surface of each ORU receptacle. Additionally, a C-H rating was 
collected after each experimental run to provide a subjective assessment of task 
difficulty. 

3.3 Data Results 

Pearson’s (r) correlations were calculated between the dependent variables and it 
was determined that the appropriate dependent measures for analysis purposes were 
TP #1 and #2 Insertion Success rate, total time to complete the task (Task Duration), 
Cooper-Harper rating of task difficulty, and the root mean square (RMS) Forces and 
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Torques at the gripper for TP #1 and #2 Insertion., Correlations between these 
dependent variables are located in Table 3-1. 


Table 3-1. Correlation Matrix for Dependent Variables. 


Dependent 

Variables 

Cooper 

Harper 

Rating 

Task 

Duration 

TP #1 

Insertion 

Success 

TP #2 

Insertion 

Success 

TP #1 
Insertion 
RMS Forces 

TP #2 
Insertion 
RMS Forces 

TP #1 

Insertion RMS 
Torques 

Task Duration 

0.634* 







TP#1 Insertion 
Success 

-0.003 

-0.076 






TP #2 Insertion 
Success 

-0.158 

-0.062 

-0.026 





TP #1 Insertion 
RMS Forces 

0.130 

0.079 

-0.090 

0.092 




TP #2 Insertion 
RMS Forces 

0.174 

0.287* 

0.020 

0.048 

0.626* 



TP #1 Insertion 
RMS Torques 

0.1406 

0.070 

-0.149 

0.045 

0.728* 

0.543* 


TP #2 Insertion 
RMS Torques 

0.284* 

0.089 

-0.110 

0.025 

nnm 

0.596* 

0.650* 


*1 -Tailed Significance at 0.001 

TP #1 and #2 Insertion Success rates were not correlated to each other or to the 
other dependent variables; thus, they were analyzed separately using univariate 
statistical procedures. Because this is attribute data (values are coded as 0 and 1) , 
Pearson Chi-Square statistics 7 (with Yates correction) were calculated. Task Duration 
and C-H rating were correlated to each other but not to the other dependent variables; 
thus, these variables were analyzed together using Multivariate Analysis of Variance 
(MANOVA) statistical procedure. 8 The RMS Force and Torque variables were 
correlated and should be analyzed together using multivariate statistical procedures. 
However, all four dependent variables could not be analyzed together because of 
limitations in the statistical analysis software used. Therefore, two MANOVA statistical 
procedures were run. One analysis used the RMS Force and Torque at the end 
effector for the TP #1 Insertion, and a separate analysis was used with the RMS Force 
and Torque at the end effector for the TP #2 Insertion. Because of the increased error 
associated with running multiple tests with correlated variables, a more stringent p- 
value of 0.01 was required to accept the difference as statistically significant. A 
significance level of 0.05 was used for the other procedures. 


7 Hicks, C. R. (1973). Fundamental concepts in the design of experiments (2nd ed.). NY: Holt, 
Rinehart and Winston. 

8 Chatfield, C. and Collins, A. J. (1980). Introduction to multivariate analysis. London: Chapman and 
Hall. 
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Kolmogorov-Smirnov Goodness of Fit tests 9 were run on these dependent 
variables. The results of these analyses indicated that the distributions of these 
dependent variables are not significantly different from the normal distribution. 

The data analysis results are presented in the following sections grouped by 
research issues (independent variables), instead of being grouped according to the 
statistical test procedures run. Appendix E contains statistical tables with test statistics 
and significance level per dependent variable. 

3.3.1 Between Subjects Effects 

Pearson Chi-Square statistics showed no statistically significant between-subjects 
effect on ability to successfully perform TP #1 Insertion (Chi-Square statistic = 1 8.96, p 
= 0.062) or TP #2 Insertion (Chi-Square statistic - 12.20, p = 0.349). MANOVA results 
showed significant between-subjects effects for: 

• Task Duration and C-H rating (Pillai's Trace statistic = 0.91 8, p - 0.000; Task 
Duration F = 57.265, p = 0.000; C-H rating F = 110.155, p = 0.000); 

• RMS Force and Torque levels for TP #1 Insertion (Pillai's Trace statistic = 
0.90983, p = 0.000; RMS Force F = 62.4506, p = 0.000; RMS Torque F = 
12.8551, p = 0.009); and 

• RMS Force and Torque levels for TP #2 Insertion (Pillai's Trace statistic = 
0.89204, p = 0.001 ; RMS Force F = 57.5600, p = 0.000; RMS Torque F = 
16.5296, p = 0.005). 

3.3.2 Reference Frame Condition 

Pearson Chi-Square statistics showed no statistically significant effect of reference 
frame on the ability of the subjects to successfully perform TP #1 Insertion (Chi-Square 
statistic = 1 .48, p = 0.224) or TP #2 Insertion (Chi-Square statistic = 2.53, p = 0.1 1 2). 
Reference frame had a statistically significant effect on Task Duration (F = 13.849; p = 
0.004). The average Task Duration according to reference frame condition is graphed 
in Figure 3-5. As illustrated, the World reference frame required approximately 43% 
more time than the End Effector reference frame. 

The Task Duration data was divided into four discrete task segments to better 
understand the effect of reference frame condition on amount of time required to 
complete the task. Two of the task segments are time to insert the ORU in each 
receptacle, referred to as TP #1 Insertion Duration and TP #2 Insertion Duration. The 
remaining two task segments are time to slew between the two task panels. Slew #1 
Duration refers to the time required to slew from Task Panel #2 to Task Panel #1 . 

Slew #2 Duration refers to the time required to slew from Task Panel #1 back to Task 
Panel #2. (refer to Figure 3-1 for the configuration of PFMA and task panel). 


9 Norusis,M. J. (1986). SPSS/PC+ for the IBM PC/XT/AT. Chicago: SPSS, Inc. 
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Figure 3-5. Task Duration per Reference Frame. 

Figure 3-6 shows the average time required for each task segment by reference 
frame condition. As shown by this figure, there is no difference between reference 
frame conditions in the amount of time required to complete Slew #2 or TP #1 
Insertion. This result is understandable because both reference frames were aligned 
when the end effector was pointed at the destination — namely, TP #1 . The difference 
between the two reference frame conditions is due to Slew #1 (from TP #1 to TP #2) 
and TP #2 Insertion requiring more time with the World reference frame versus the End 
Effector reference frame. The destination for Slew #1 and TP #2 Insertion is yawed 
approximately 45-degrees to the left of the centerline (X axis). Therefore, when the 
gripper was oriented towards TP #2, the World and End Effector references were not 
aligned. Thus, the World reference frame required more time to perform those task 
segments where the reference frame were not aligned with the end effector. 

A significant effect of reference frame on C-H rating was also found (F = 18.190; p = 
0.002). Figure 3-7 shows the average C-H rating per reference frame condition. As 
shown by this figure, the World reference frame was perceived as more difficult than 
the End Effector reference frame. Since the World reference frame was perceived as 
more difficult, it can be hypo thesiz ed t hat the World reference frame required more 
time to perfor m the ta sk b ecause it required mor e cognitive processing. 

There was no statistically significant effect of reference frame for TP #1 or #2 
Insertion RMS Force and Torque levels (TP #1 Pillai's Trace value = 0.041 , p = 0.881 ; 
TP #2 Pillai's Trace value = 0.541, p = 0.097). 
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Figure 3-6. Task Duration by Task Segments. 
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Figure 3-7. Cooper Harper Rating per Reference Frame. 

Post test questionnaire results showed that 1 1 of the 12 operators (91 .7%) 
preferred using the End Effector reference frame for performing the ORU replacement 
task. The subjects were also asked to rate how strongly they preferred one reference 
frame over the other reference frame. The strength of preference was rated on a scale 
from "0" to "3" with "0" for no, "1" for slight, "2" for moderate, and ”3" for strong 
preference. The average preference rating was 2.4, indicating a moderate to strong 
preference for the End Effector reference frame. The one person who preferred the 
World reference frame rated his/her preference strength as a 3. 
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The post-test questionnaire asked the operators to indicate what reference frame 
they preferred for the two different task segments— slewing and fine alignment— and 
the strength of their preference. Nine out of 12 operators preferred the End Effector 
reference frame for slewing and the average degree of preference was a 2 (moderate). 
The remaining 3 operators who preferred the World reference frame for slewing had a 
average preference strength rating of 1.3 (slight to moderate). The difference between 
the two preference strength ratings were not statistically significant (F = 1 .15; p = 0.31 ). 
All of the operators preferred the End Effector reference frame for fine alignment and 
the average strength of this preference was 2.7 (moderate to strong). 

The operators were also asked to indicate why they preferred one reference frame 
over the other. In general, the operators indicated that the End Effector reference 
frame was easier to use, felt more natural, required less mental effort, and was more 
responsive. The reason for preferring the World reference frame was that it allowed 
pre-planning and provided an easier mental model. 

3.3.3 Impedance Level Condition 

Pearson Chi-Square statistics showed no statistically significant effect of 
Impedance level on TP #1 or TP #2 Insertion Success rate (TP #1 Chi-Square statistic 
= 3.05, p = 0.081 ; TP #2 Chi-Square statistic = 0.60, p = 0.438). No statistically 
significant effects due to impedance level was found for either Task Duration or C-H 
rating (Pillai's Trace value = 0.389, p = 0.109). There was no statistically significant 
effects of impedance level for TP #1 or #2 Insertion RMS Force and Torque levels at 
the gripper (TP #1 Pillai's Trace value = 0.332, p = 0.298; TP #2 Pillai's Trace value = 
0.337, p = 0.292). 

The operators were asked on the post-test questionnaire if they were able to 
distinguish between the two levels of impedance. Five of the 12 operators (41.7%) 
indicated that the difference between the two levels was noticeable. Of the 5 subjects 
who noticed the difference in impedance levels, 3 indicated a preference for the "stiff” 
impedance for performing the task and gave an average rating of the strength of their 
preference as a 2 (moderate). The 2 subjects who preferred the "soft" impedance also 
gave a 2 as the average strength of their preference. 

The operators were also asked to indicate which impedance level was preferred for 
the different task segments. Three of the 5 operators preferred the "stiff" condition for 
slewing and gave an average preference strength rating of 1.33 (none to slight). The 2 
operators who preferred the "soft" condition for slewing gave an average preference 
strength rating of 1.5 (1 subject rated no preference and one rated a slight preference). 
For fine alignment tasks, 4 of the 5 preferred the "soft" impedance and gave an 
average preference strength rating of 3 (strong). The 1 subject that preferred the stiff 
condition gave an average strength rating of 2 (moderate). 

The subjects were asked to indicate the reason for preferring one impedance level 
over the other. The main reason expressed for preferring the "stiff" impedance 
condition was that it was more responsive to small motions without overshooting. The 
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main reason for preferring the "soft" impedance condition was that it had a more 
natural, relaxed feel. 

3.3.4 Time Delay Effect 

Pearson Chi-Square tests indicated no statistically significant effect of time delay 
on the ability of the subjects to successfully perform TP #1 or #2 Insertion (TP #1 Chi- 
Square statistic =4.62, p = 0.1 00; TP #2 Chi-Square statistic = 3.90, p = 0.1 42). A 
MANOVA analysis found that time delay had a statistically significant effect on Task 
Duration and C-H rating (Pillai's Trace value = 0.855, p = 0.000; Task Duration F = 
28.703; p = 0.000; C-H F = 24.1 77; p = 0.000). 

The average Task Duration according to time delay condition is shown in Figure 3- 
8. As shown in this graph, the time required to complete the task increased as delay 
increased. The increase between no time delay and 1 second of time delay 
(approximately 45% increase) is much larger than the increase between 1 second and 
2 seconds (approximately 16% increase). Thus, the increase of time delay (up to 2 
seconds) appears to degrade performance less than the addition of time delay. 
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Figure 3-8. Task Duration by Time Delay Condition. 

The Task Duration data was divided into the four task segments defined in Section 
3.3.2. Figure 3-9 shows the average amount of time to complete each segment per 
time delay condition. As shown by this figure, the increase in time delay affected the 
amount of time required to complete Slew #1 the most. Slew #1 was the first slew that 
the operators performed. It is feasible that this task segment was affected more by time 
delay than the other segments simply because of the effect of learning. Once the 
operators became accustomed to the presence of time delay, as the task progressed, it 
may have influenced performance less. 
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Figure 3-9. Task Segment Duration by Time Delay Condition. 

As already mentioned, the MANOVA showed a statistically significant effect of time 
delay on C-H rating. Figure 3-10 shows the average C-H rating per time delay 
condition. As shown by this figure, the operators' perception of task difficulty increased 
as the time delay increased. Since the subjects required more time to perform the task 
and the task was perceived as more difficult as time delay increased, it can be 
hypothesized that more cognitive processing was required as time delay increased. 

There were no statistically significant effects of time delay for TP #1 or #2 Insertion 
RMS Force and Torque levels (TP #1 Piliai’s Trace statistic = 0.241 , p = 0.854; TP #2 
Pillai's Trace statistic = 0.831 , p = 0.076). 

All 12 of the subjects indicated on the post-test questionnaire that they were able to 
distinguish between the time delay conditions. The operators were queried as to the j 

type of strategy they used to accommodate the time delay. In general, the operators 
indicated that they adopted a move and wait strategy, and made smaller, more 
precise, single axis inputs. Four of the operators adopted a strategy where they 
actually estimated or timed the delay and paced the lag between their inputs to equal 
the delay. The operators also indicated that the task was more time consuming and 
required more thought with time delay than without time delay. The subjects were also j 

asked to recommend methods for making the task easier to perform in the presence of j 

time delay. All of the suggestions inferred the use of a predictive display. 

3.3.5 Interaction Effects 

The results of MANOVA analyses indicated no two- or three-way interaction effects 
for Task Duration, C-H rating, TP #1 Insertion RMS Force and Torque levels, or TP #2 
Insertion RMS Force and Torque levels. See Appendix F for the Pillai's Trace and p 
values. 


34 




Figure 3-10. Cooper Harper Rating by Time Delay Condition. 

The post-test questionnaire asked the subjects to indicate the reference frame that 
they would prefer under the different combinations of delay and impedance condition. 
Regardless of the combination of conditions, 9 of the subjects preferred the End 
Effector reference frame and 1 preferred the World. One subject preferred the World 
reference frame when there was no time delay and preferred the End Effector 
reference frame in the presence of time delay. No clear trends emerged when the 
subjects were asked to selected the preferred impedance condition per combination of 
reference frame and time delay conditions. 

3.3.6 Predictors of Operator Performance 

Each subject completed a background questionnaire that asked for information 
such as how many years s/he has had a drivers’ license, previous experience playing 
video games, and experience controlling remotely controlled models. (The 
background questionnaire is located in Appendix B). Stepwise regression analyses 
were performed to determine if there were any background characteristics of the 
subjects that would predict the amount of time required to perform the task (T ask 
Duration) and the perceived difficulty of the task as measured by a C-H rating (the two 
dependent variables that were affected by the experimental conditions). 

The results of the regression analysis on Task Duration yielded no factors that 
predicted performance. The results of the regression analysis on C-H rating yielded 
two factors that predicted subjective assessment of task difficulty. The number of years 
that the operator had a drivers license accounted for 58% of the variance in the data; 
the addition of the person’s college major into the regression equation accounted for a 
total of 789% of the variation in the data. There was a negative correlation between 
the number of years that the person has had a driving license and the C-H rating; thus, 


35 


the longer the person had a drivers license, the easier they tended to rated the task 
difficulty. With regard to college major, subjects whose major was Management 
Information Systems (MIS) tended to rate the task easier than those subjects in other 
disciplines (e.g., physics, math, and engineering). 


3.4 Discussion 

High between-subjects variability was noted for all of the dependent variables, 
except ability to successfully insert the ORU into the task panel receptacles. This 
points out the need for using a within-subjects experimental design for research of this 
type to account for the between-subjects variability. 

The End Effector reference frame required approximately 43% less time than the 
World reference frame to perform the ORU insertion task and all but one subject had, 
on the average, a moderately strong preference for the End Effector reference frame. 
When the task duration data was viewed by task segments, it was observed that the 
World reference frame required more time to perform those task segments where the 
reference frame was not aligned with the end effector. The subjective responses 
showed that the majority of the operators preferred the End Effector reference frame for 
the stewing portion of the task, all of them preferred it for the fine alignment portion. 

This indicates the importance of controlling a manipulator about a point that has 
relevance to the task being performed. 

The World reference frame was rated as significantly more difficulty than the End 
Effector reference frame. In light of this fact, it is possible that the World reference 
frame required more time because it required more cognitive processing. This is 
supported by the post-test questionnaire responses where the subjects indicated that 
the End Effector reference frame was easier to use and required less mental effort. 

No objective performance differences were observed between the impedance 
conditions and no clear subjective results emerged. Only 5 of the 12 (41.7%) 
operators Indicated that they noticed a diff eren ce between the two conditions. Of 
those 5 subjects, 3 preferred the "stiff" condition and 2 preferred the "soft" condition. 
Additionally, the strength of their preference was "none" to "slight." No differences may 
have been noted because there was not enough variability between the two 
conditions. : - i ~ “ 

Time delay had a statistically significant effect on time to complete the task, such 
that there was approximately a 45% in crea se from the no time delay condition to 1 
second of time delay and approximately a 16% increase from the 1 second to 2 
second time delay conditions. Analysis of the different task segments showed that 
Slew #1— slew from TP #1 to TP #2— was affected the most by time delay. It is 
possible that this task segment was most affected simply because of the effect of 
learning, since this was the first task segment. 

Not only did the amount of time required to perform the task increase with time 
delay, but also the operator's subjective assessment of task difficulty (C-H rating) 
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increased. This is in concert with the operators' responses that the task was more time 
consuming and required more thought with time delay than without any time delay. 
When queried as to the type of strategy used to accommodate the time delay, the 
operators stated that they adopted a move and wait strategy and made smaller, more 
precise, single axis inputs. The subjects also indicated that the a predictive display 
would make the task easier to perform in the presence of time delay. 

None of the background information gathered from the subjects helped predict task 
performance as measured by Task Duration. However, the number of years that the 
person has had a drivers license and s/he’s college major did predict subjective 
assessment of task difficulty, as given by a C*H rating. In general, the longer the 
person has had a drivers license, the easier s/he rated the task and MIS majors rated 
the task easier than other majors (e.g., physics, math, and engineering). 
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Appendix A - Informed Consent Form 


INFORMED CONSENT FORM 

Teleoperation Study: Proto-Flight Manipulator Arm at Marshall Space 

Flight Center (MSFC) 


I agree to participate in the above study being conducted by Martin Marietta 
Astronautics Group in conjunction with Marshall Space Flight Center, involving the 
operation of the Proto-Flight Manipulator Arm (PFMA). The research team has 
provided a general overview of the tasks that I will be asked to perform. I understand 
that participation is voluntary and that I may discontinue the study at any time. All 
information will be collected on an anonymous basis, which will ensure confidentiality. 


I understand that I will be paid $10.00 per hour for my participation in this study. I 
also understand that payment will be made by means of a check that will be mailed to 
the address provided below. 


SIGNATURE: 
NAME (PRINTED): 


DATE: 


ADDRESS: 



•WttwwwM* mm 


3 9 PRECEDING PAGE BLANK NOT FILMED 


Appendix B - Background Questionnaire 


QqL 


Subject # 


BACKGROUND INFORMATION 


Age: 

Years Months 


4-7 


What is your class standing? (check one) 


1 . 

Freshman 

2. 

Sophomore 

3. 

Junior 

4. 

Senior 

5. 

Master's level graduate student 

6. 

Ph.D. level graduate student 


Whafisynrmapr? 

What is your area of specialization or emphasis (e.g., robotics, 
artificial irteioenae) ? 

Do you have a driver's license? (check one) 

1. Yes 

2. No 

If you answered "Yes" to the previous question, how many years have 
you had a driver's license? 

Do you have a pilot's license? (check one) 

1. Yes 

2. NO 

If you answered "Yes" to the previous question, approximately how many 
hours have you logged as pilot-in command or as co-pilot? - 

Do you play videogames that use a joystick or trackball? (check one) 


1. Yes 

2. No 

If you answered "Yes" to the previous question, how many hours per month 
(on the average) do you normally play? 

Have you overflown radio-controlled models (e g., boats, airplanes, cars)? 
(check one) 

1. Yes 


9 


11 

13 

15 


17-18 

20 


22-25 

27 


29-31 

33 


40 


2. No 

r 

If you answered "Yes" to the previous question: 
Approximately how many hours have you logged? 
What type of controller did you use? (check one) 

1 . Joystick 

2. Wheel 

3. Other- Reasespedy- 


Have you ever run the Proto-Flight Manipulator Arm (PFMA) with a 
hand controller before? (check one) 

1. Yes 

2. No 

If you answered "Yes" to the previous question: 

How long ago was your experience? Yeas 

How many total hours did you spend running the PFMA? 

Have you ever used a hand controller like the one shown to you? (check one) 

1 . Yes 

2. No 

If you answered "Yes" to the previous question: 

How long ago was your experience? Years 

How many total hours dfcJ you spend using the hand controller? 

For what purpose cSd you use the hand controller? 


Months 


Months 
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44-48 

50-53 

55 


57-61 

63-66 

68 


THANK YOU!!! 


DIFFICULTY LEVEL OPERATOR OEMANO LEVEL RATWtt 


Appendix C - Cooper-Harper Rating Scale 
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OPERATOR DECISIONS 








Appendix D - Post-Test Questionnaire 


Subject #_ 


POST-TEST QUESTIONNAIRE 

Please answer the following questions as thoroughly as possible. If you require more space, use the back 
of the page. If you have any questions, please ask one of the test conductors for assistance. 

The questions ask you to consider not only the overall task, but also different parts of the task you 
performed. The parts you are asked to consider are the Slewing and Fine Alignment task parts. Slewing 
refers to such motions as moving the box from one location to another location. Fine Alignment refers to 
such motions as placing the box in or removing it from the task panel. Slewing motions consist of larger, 
less precise movements. Fine Alignment motions consist of smaller, more precise movements. 

REFERENCE FRAME 

World reference frame refers to the reference frame that was located above and behind the 
manipulator arm. 

End Effector reference frame refers to the reference frame that was located near the manipulator's 
end effector. 


Overall Slewing Fine Alignment 

Task 

Which reference 
frame do you 
prefer? Write 
World or End 
Effector in the 

blanks provided. — 

How strongly is 
your preference? 

Use the following 
numbers to indicate 
level of preference: 

0= none 

1 = slight 

2 = moderate 

3- strong 
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Why do you 
prefer this 
reference frame? 
Example responses: 
"easy to use" 

"felt natural" 

"more controllable" 
"more responsive" 


IMPEDANCE 

Overall Slewing Fine Alignment 

Task 

Which impedance 
do you prefer? 

Write Soft or Stiff 
in the blanks 

provided. . — 

How strongly is 
your preference? 

Use the following 
numbers to indicate 
level of preference: 

0= none 
1 = slight 
2 = moderate 

3= strong 

Why do you 
prefer this 

impedance? 

Example responses: 

"easy to use" — 

"felt natural" 

"more controllable" 

"more responsive" 
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TIME DELAY 

Were the different time delays noticeable? (check one) 

1. No 

2. Yes 

What type of strategy did you use to accommodate the time delay? 


What difference did the time delay make in terms of the way that you performed the task? 


What could have been done to make it easier for you to perform the task with time delay? 


Please fill out the following matrices as best as you can. 


Under the following combinations, which reference frame did you prefer for the overall task? (write World 
or End Effector in each cell) 


1 

Soft Impedance 

Stiff Impedance 











Under the following combinations, which impedance did you prefer for the overall task? 
(write Soft or Stiff in each cell) 


1 

World Reference Frame 

End Effector Reference Frame 
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Appendix E — Statistical Tables per Dependent Variable(s) 


P-values that are less than or equal to 0.05 have an asterisk * behind them. 

Table E-1. Pearson Chi-Square Table for TP 411 Insertion Success. 



Table E-2. Pearson Chi-Square Table for TP 442 Insertion Success. 


FACTOR 

CHI-SQUARE 

VALUE 

p- VALUE 

Between Subjects 

1220 

0.349 

Within Subjects 



Reference Frame 

2.53 

0.112 

Impedance 

0.60 

0.438 

Time Delay 

3.90 

0.142 
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Table E-3. MANOVA Table for Task Duration and CH Rating. 


FACTOR 

■uiisjmi 

Hflil 

P- 

VALUE 


0318 


0.000* 

Duration 


57265 

0.000* 

Cooper Harper 


110.155 

0.000* 





Main Effects 




Reference Frame 

0.649 



Duration 


13.849 

PB 

Cooper Harper 


18.190 


Impedance 

0.389 


0.109 | 

Time Delay 

0B55 


ululuBi 

Duration 


28.703 


Cooper Harper 


24.177 

0.000* 

Interaction Effects 




Frame x Impedance 

0.078 


0.695 

Frame x Delay 

0.362 


0.471 

Impedance x Delay 

0251 


0.684 

Frame x Impedance x Delay 

0.574 


0.151 


Table E-4. MANOVA Table for RMS Force and Torque for ORU # 1 
Insertion. 


FACTOR 

PILLAI'S 
TRACE VALUE 

F 

VALUE 

P- 

VALUE 


0910 



RMS Force 


62.451 

EmI^I 

RMS Torque 


12.855 






! Main Effects 




Reference Frame 

0.041 


0.881 

Impedance 

0.332 


0298 

Time Delay 

0241 


0.854 

Interaction Effects 




Frame x Impedance 

0.385 


0232 

Frame x Delay 

0.725 


0.185 

Impedance x Delay 

0268 


0.823 

Frame x Impedance x Delay 

0.405 


0.641 
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Table E-5. MANOVA Table for RMS Force and Torque for ORU #2 
Insertion. 


FACTOR 

PILLAI'S 
TRACE VALUE 

F 

VALUE 

P- 

VALUE 


0B92 

57.560 

0.001* 

RMS Force 


0.000* 

RMS Torque 


16.530 

0.005* 





Main Effects 



0.097 

Reference Frame 

0.541 


Impedance 

0.337 


0292 

Time Delay 

0.831 


0.076 

Interaction Effects 



0.523 

Frame x Impedance 

0.194 


Frame x Delay 

0.392 


0.660 

Impedance x Delay 

0.531 


0.454 

Frame x Impedance x Delay 

0.721 


0.190 
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